69 research outputs found
Multi-granularity Item-based Contrastive Recommendation
Contrastive learning (CL) has shown its power in recommendation. However,
most CL-based recommendation models build their CL tasks merely focusing on the
user's aspects, ignoring the rich diverse information in items. In this work,
we propose a novel Multi-granularity item-based contrastive learning (MicRec)
framework for the matching stage (i.e., candidate generation) in
recommendation, which systematically introduces multi-aspect item-related
information to representation learning with CL. Specifically, we build three
item-based CL tasks as a set of plug-and-play auxiliary objectives to capture
item correlations in feature, semantic and session levels. The feature-level
item CL aims to learn the fine-grained feature-level item correlations via
items and their augmentations. The semantic-level item CL focuses on the
coarse-grained semantic correlations between semantically related items. The
session-level item CL highlights the global behavioral correlations of items
from users' sequential behaviors in all sessions. In experiments, we conduct
both offline and online evaluations on real-world datasets, verifying the
effectiveness and universality of three proposed CL tasks. Currently, MicRec
has been deployed on a real-world recommender system, affecting millions of
users. The source code will be released in the future.Comment: 17 pages, under revie
Reweighting Clicks with Dwell Time in Recommendation
The click behavior is the most widely-used user positive feedback in
recommendation. However, simply considering each click equally in training may
suffer from clickbaits and title-content mismatching, and thus fail to
precisely capture users' real satisfaction on items. Dwell time could be viewed
as a high-quality quantitative indicator of user preferences on each click,
while existing recommendation models do not fully explore the modeling of dwell
time. In this work, we focus on reweighting clicks with dwell time in
recommendation. Precisely, we first define a new behavior named valid read,
which helps to select high-quality click instances for different users and
items via dwell time. Next, we propose a normalized dwell time function to
reweight click signals in training, which could better guide our model to
provide a high-quality and efficient reading. The Click reweighting model
achieves significant improvements on both offline and online evaluations in a
real-world system.Comment: 5 pages, under revie
Improve Transformer Pre-Training with Decoupled Directional Relative Position Encoding and Representation Differentiations
In this work, we revisit the Transformer-based pre-trained language models
and identify two problems that may limit the expressiveness of the model.
Firstly, existing relative position encoding models (e.g., T5 and DEBERTA)
confuse two heterogeneous information: relative distance and direction. It may
make the model unable to capture the associative semantics of the same
direction or the same distance, which in turn affects the performance of
downstream tasks. Secondly, we notice the pre-trained BERT with Mask Language
Modeling (MLM) pre-training objective outputs similar token representations and
attention weights of different heads, which may impose difficulties in
capturing discriminative semantic representations. Motivated by the above
investigation, we propose two novel techniques to improve pre-trained language
models: Decoupled Directional Relative Position (DDRP) encoding and MTH
pre-training objective. DDRP decouples the relative distance features and the
directional features in classical relative position encoding for better
position information understanding. MTH designs two novel auxiliary losses
besides MLM to enlarge the dissimilarities between (a) last hidden states of
different tokens, and (b) attention weights of different heads, alleviating
homogenization and anisotropic problem in representation learning for better
optimization. Extensive experiments and ablation studies on GLUE benchmark
demonstrate the effectiveness of our proposed methods
Learning from All Sides: Diversified Positive Augmentation via Self-distillation in Recommendation
Personalized recommendation relies on user historical behaviors to provide
user-interested items, and thus seriously struggles with the data sparsity
issue. A powerful positive item augmentation is beneficial to address the
sparsity issue, while few works could jointly consider both the accuracy and
diversity of these augmented training labels. In this work, we propose a novel
model-agnostic Diversified self-distillation guided positive augmentation
(DivSPA) for accurate and diverse positive item augmentations. Specifically,
DivSPA first conducts three types of retrieval strategies to collect
high-quality and diverse positive item candidates according to users' overall
interests, short-term intentions, and similar users. Next, a self-distillation
module is conducted to double-check and rerank these candidates as the final
positive augmentations. Extensive offline and online evaluations verify the
effectiveness of our proposed DivSPA on both accuracy and diversity. DivSPA is
simple and effective, which could be conveniently adapted to other base models
and systems. Currently, DivSPA has been deployed on multiple widely-used
real-world recommender systems
Personalized Prompt for Sequential Recommendation
Pre-training models have shown their power in sequential recommendation.
Recently, prompt has been widely explored and verified for tuning in NLP
pre-training, which could help to more effectively and efficiently extract
useful knowledge from pre-training models for downstream tasks, especially in
cold-start scenarios. However, it is challenging to bring prompt-tuning from
NLP to recommendation, since the tokens in recommendation (i.e., items) do not
have explicit explainable semantics, and the sequence modeling should be
personalized. In this work, we first introduces prompt to recommendation and
propose a novel Personalized prompt-based recommendation (PPR) framework for
cold-start recommendation. Specifically, we build the personalized soft prefix
prompt via a prompt generator based on user profiles and enable a sufficient
training of prompts via a prompt-oriented contrastive learning with both
prompt- and behavior-based augmentations. We conduct extensive evaluations on
various tasks. In both few-shot and zero-shot recommendation, PPR models
achieve significant improvements over baselines on various metrics in three
large-scale open datasets. We also conduct ablation tests and sparsity analysis
for a better understanding of PPR. Moreover, We further verify PPR's
universality on different pre-training models, and conduct explorations on
PPR's other promising downstream tasks including cross-domain recommendation
and user profile prediction
- …